Various different types of collaborative and/or distributed computing are known by various terms such as “Grid Computing”, “Cluster Computing” and “Cloud Computing” which may overlap with each other to some extent.
Grid computing may be thought of as the application of several computers to a single problem at the same time, the problem being sufficiently demanding of processing time or “power” (due to it requiring a very high number of computer processing cycles, or access to large amounts of data) that it may only realistically be completed within a reasonable amount of time by sharing it between several computers. Grid computing depends on software to divide and apportion pieces of a program among several computers, sometimes up to many thousands. Grid computing can also be thought of as distributed and large-scale cluster computing, as well as a form of network-distributed parallel processing. It can be small—confined to a network of computer workstations within a corporation, for example—or it can be a large, public collaboration across many companies or networks.
Grid computing may be thought of as a form of distributed computing whereby a cluster of networked, loosely-coupled computers effectively act together as a single, virtual, super-computer in order to be able to perform very large tasks. This concept has been applied to computationally-intensive scientific, mathematical and academic problems, and has been used for applications such as drug discovery, economic forecasting, seismic analysis, back-office data processing etc.
Grid computing can be distinguished from conventional cluster computing systems in that grids tend to be more loosely coupled, heterogeneous, and geographically dispersed. The term “cloud computing” meanwhile relates to collaborative or distributed Internet (“cloud”) based development and use of computer technology, often based on the idea that processing and data storage may be done “in the cloud” from the point of view of the user or client, who is thus able to perform the same or similar actions irrespective of location, and from any of a variety of different “thin” user terminals with little more than basic browser functionality provided the user terminal currently in use is in communication with the “cloud”.
Evening the load over a population of processing units may be an important element of grid, cloud, cluster, or other types of collaborative computing, as it is in this way that idling resources can be used to relieve overloaded ones, increasing the overall throughput of the whole population, and, by spreading peaks in demand more equally between units, decreasing the processing requirements that may be expected of any one unit.
With grid computing for example, in most cases, the workload distribution is coordinated centrally by a resource broker. However, because this approach doesn't scale well, researchers have been looking at ways of achieving global load-balancing through decentralised workload distribution techniques.
One of these decentralised techniques is elegantly simple and involves mimicking the natural process of chemical diffusion. Essentially, each processing node in the system monitors the length of its local queue relative to that of some predefined potential “collaborators”. Periodically, a fraction of the difference is transferred between neighbours so as to even the load. For instance if node X has 10 jobs queuing (hereafter written X[10]) and node Y has only 4 (Y[4]), then k(10-4) jobs are transferred from X to Y, where k is a parameter representing the maximum fraction of the total workload transferable in a single attempt. For instance, if k=1/3, in the above example, 6/3=2 jobs will be transferred, resulting in the configuration X[8],Y[6] which is closer to a homogeneous workload distribution. Assuming Y has a second collaborator Z with, say, a queue of 3 (Z[3]), then the next step could result in a configuration of X[8],Y[5],Z[4], then X[7],Y[6],Z[4], i.e. the workload is progressively distributed more evenly, without the need for any central planning. Variants of this model include “chemotaxis”, in which a second signal representing the availability of local resources also diffuses, so as to “guide” the workload towards the region of the system where it will be most rapidly processed, through asymmetrical diffusion. This is explained in more detail in the following article: Canright, G., Deutsch, A., Urnes, T.: “Chemotaxis-Inspired Load Balancing”, COMPLEXUS, VOL 3; NUMBER 1-3, pages 8-23 (2006).